The Matrix Chain Algorithm to Compile Linear Algebra Expressions

occur in information theory [1], and regularization [2], respectively. Given such expressions, we are interested in the automatic generation of code that is at least as fast and as numerically stable as what an expert would produce. Conceptually, the problem is similar to how compilers cast scalar expressions in terms of the available instruction set. The corresponding problem for linear algebra expressions (involving matrices) is much more challenging, and requires expertise in both numerical linear algebra and high-performance computing. On the one hand, one wants to take advantage of highly optimized building blocks for matrix operations, such as those as provided by the BLAS [3] and LAPACK [4] libraries. On the other hand, transformations based on associativity, commutativity and distributivity play an essential role. Further complication comes from the fact that matrices frequently have structures and properties that can be exploited both to transform—and thus simplify— expressions, and to evaluate them more efficiently. The application of this kind of knowledge affects not only the computational cost, but also the necessary amount of storage space, and numerical accuracy. At the moment, there are two options for dealing with complex matrix expressions. One either has to map the expressions to kernels manually, or use highlevel programming languages and environments such as Matlab and R. The first option involves a lengthy, error-prone process that usually requires a numerical linear algebra expert. The second option, using highlevel programming languages, is a very convenient alternative in terms of productivity, but rarely leads to the chain → factor · chain | factor (1) factor → op | op | op | op (2)